Long-context models (LCMs) have made remarkable strides in recent years, offering users great convenience for handling tasks that involve long context, such as document summarization. As the community increasingly prioritizes the faithfulness of generated results, merely ensuring the accuracy of LCM outputs is insufficient, as it is quite challenging for humans to verify the results from the extremely lengthy context. Yet, although some efforts have been made to assess whether LCMs respond truly based on the context, these works either are limited to specific tasks or heavily rely on external evaluation resources like GPT4.In this work, we introduce L-CiteEval, a comprehensive multi-task benchmark for long-context understanding with citations, aiming to evaluate both the understanding capability and faithfulness of LCMs. L-CiteEval covers 11 tasks from diverse domains, spanning context lengths from 8K to 48K, and provides a fully automated evaluation suite. Through testing with 11 cutting-edge closed-source and open-source LCMs, we find that although these models show minor differences in their generated results, open-source models substantially trail behind their closed-source counterparts in terms of citation accuracy and recall. This suggests that current open-source LCMs are prone to responding based on their inherent knowledge rather than the given context, posing a significant risk to the user experience in practical applications. We also evaluate the RAG approach and observe that RAG can significantly improve the faithfulness of LCMs, albeit with a slight decrease in the generation quality. Furthermore, we discover a correlation between the attention mechanisms of LCMs and the citation generation process.

Fox News Flash top headlines are here. Check out what's clicking on Foxnews.com. A Pennsylvania man who uses drones to try to locate wounded deer shot by hunters so they can retrieve their carcasses has been convicted of violating state hunting laws. Joshua Wingenroth, 35, of Downingtown, plans to appeal the verdicts handed down Thursday by Lancaster County District Judge Raymond Sheller. The case apparently marked the first time anyone has been cited and tried in Pennsylvania for using a drone to recover a dead game animal and it hinged on whether Wingenroth was involved in hunting as defined by state law.

Fox News Flash top sports headlines are here. Check out what's clicking on Foxnews.com. A Pennsylvania man could face up to four years in prison after he was charged in a federal criminal complaint for illegally flying a drone over the Baltimore Ravens stadium during the AFC Championship last month, causing an unusual delay of game. The U.S. Attorney's Office for the District of Maryland announced the charges on Monday, alleging that Matthew Hebert, 44, violated a temporary flight restriction placed on M&T Bank Stadium when he flew a drone over the area during the NFL game. Zay Flowers of the Ravens makes a catch for touchdown during the AFC Championship game against the Kansas City Chiefs at M&T Bank Stadium on Jan. 28, 2024, in Baltimore.

During an address Thursday, President Joe Biden claimed he taught "political theory" at the University of Pennsylvania. An Altoona, Pennsylvania man has been arrested after allegedly having a sexual relationship with a teenage girl he met on the popular dating app, Tinder, according to reports. An NBC station out of Johnstown-Altoona, Pennsylvania reported that state police spoke with the 14-year-old girl on Sept. 19. During the conversation, the girl reportedly told police she met Steven Ellis, 32, on Tinder after creating a profile that made her appear older. In court documents, police said they learned the teenager and Ellis sent each other explicit messages and photos.

Fox News Flash top headlines for Sept. 18 are here. Check out what's clicking on Foxnews.com A man arrested in June for allegedly dropping explosive devices in an eastern Pennsylvania community via a drone was targeting his ex-girlfriend's home, according to prosecutors. Jason Muzzicato, 44, was taken into custody in Washington Township in Northampton County by the FBI and local authorities after he was linked to the explosions. Evidence at his home and his business, Bangor Motor Works, tied him to several explosions in the township since March.

Fox News Flash top headlines for August 29 are here. Check out what's clicking on Foxnews.com A Pennsylvania court ruled Tuesday that making a "gunlike hand gesture" is a crime after a man made the hand motion during an argument with his neighbor -- an act which reportedly made several nearby residents nervous and prompted a call to police. Stephen Kirchner, 64, made the gesture toward his neighbor in Manor Township in June 2018, according to surveillance video. Kirchner, walking alongside a female neighbor, "stopped, made eye contact with [the male neighbor] and then made a hand gesture at him imitating the firing and recoiling of a gun," according to court documents.